
i( United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1 450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


09/927,326 


08/13/2001 


Satoshi Ejima 


032470.01 


5540 



25944 7590 08/13/2004 

OLIFF & BERRIDGE, PLC 
P.O. BOX 19928 
ALEXANDRIA, VA 22320 



EXAMINER 



GENCO, BRIAN C 



ART UNIT 



PAPER NUMBER 



2615 

DATE MAILED: 08/13/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



w 


Application No. 


Applicant(s) 




Off icg Action SSummarv 


09/927,326 


EJIMA ET AL 




Examiner 

Brian C Genco 


Art Unit 

2615 





■- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

• If NO period for reply is specified above, the maximum statutory period will apply and wilt expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) H Claim(s) 1-38 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) 03 Claim(s) 1-38 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) S Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)ES All b)D Some * c)D None of: 

1 M Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) I3 Notice of References Cited (PTO-892) 4) O Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) d Notice of Informal Patent Application (PTO-152) 

Paper No(s)/Mail Date . 6) Q Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 6 



Application/Control Number: 09/927,326 Page 2 

Art Unit: 2615 

Examiner notes that the instant application is a continuation off of Application Serial # 
08/972,455 wherein the rejection previously presented therein is herein repeated since no 
amendments to the claims or arguments against the rejection were presented. Addition grounds 
of rejection are also presented herein bellow. 

Specification 

The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Claims 1-38 are rejected under 35 U.S.C. 102(e) as being anticipated by (USPN 
5,818,436 to Imai et al). 

In regards to claim 1, Imai discloses an information processing apparatus in which 
continuous data which may be in the form of audio or video is stored on a disk by the continuous 
data reading section. Imai further discloses a specifying section (6) which acts as a selection 
means for selecting data set from the data sets stored (col. 5, lines 3-37). Imai further discloses, 
in fig. 35, an apparatus for playing back continuous data in which audio and video data is stored 
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wherein a first audio data is represent in a first time zone (T 1) and a second audio data is 
represented in a second time (T2). The time zones are linked to overlap to produce continuous 
data (col. 22, lines 37-67). 

Regarding claim 2, Imai disclose, in fig. 1, a microphone for recording sound data, a 
camera for recording image data, and an electronic tablet for recording line drawing data (col. 4, 
lines 9-51) for inputting events. 

Regarding claim 3, Imai disclose, in fig. 1, a camera (col. 4, lines 9-20). Cameras 
inherently have light collecting means, photoelectric conversion means, a conversion means for 
converting electric signal into image data. 

Regarding claim 4, Imai disclose the pen being pressed down and making a stroke and 
thereby recording the current time and sequence of coordinate points (col. 6, lines 27-35). 

Regarding claim 5, Imai disclose, in fig. 8, a display section (8) for displaying input 
events (col. 8, lines 29-43) wherein a pointer indicates the relative time location within the 
continuous data (col. 10, lines 10-17). 

Regarding claim 6, Imai discloses adding new input events which may be represented by 
time zones (col. 22, lines 37-67) and updating time information (col. 13, lines 44-51). 

Regarding claim 7, Imai discloses that the continuous data may be digital (col. 5, lines 

5-6). 

Regarding claim 8, Imai discloses that when a new input is made, the time, an attribute if 
the type of input, and a sequence of coordinate points are written to memory (col. 6, lines 37-67). 

Regarding claim 9, Imai disclose an information processing apparatus in which 
continuous data which may be in the form of audio or video is stored on a disk by the continuous 
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data reading section. Imai further discloses a specifying section (6) which acts as a selection 
means for selecting data set from the data sets stored (col. 5, lines 3-37). Imai further discloses, 
in fig. 35, an apparatus for playing back continuous data in which audio and video data is stored 
wherein a first audio data is represent in a first time zone (TI) and a second audio data is 
represented in a second time (T2). The time zones are linked to overlap to produce continuous 
data (col. 22, lines 37-67). 

Regarding claim 10, Imai discloses that when a new input is made, the time, an attribute 
if the type of input, and a sequence of coordinate points are written to memory (col. 6, lines 
37-67). 

Regarding claim 11, Imai discloses play back only in certain periods determined by the 
user who slides the bar on the time axis (col. 10, lines 10-14). 

Regarding claim 12, Imai discloses that when a new input is made, the time, an attribute 
if the type of input, and a sequence of coordinate points are written to memory (col. 6, lines 
37-67). 

Regarding claim 13, Imai disclose an information processing apparatus in which 
continuous data which may be in the form of audio or video is stored on a disk by the continuous 
data reading section. Imai further discloses a specifying section (6) which acts as a selection 
means for selecting data set from the data sets stored (col. 5, lines 3-37). Imai further discloses, 
in fig. 35, an apparatus for playing back continuous data in which audio and video data is stored 
wherein a first audio data is represent in a first time zone (Tl) and a second audio data is 
represented in a second time (T2). The time zones are linked to overlap to produce continuous 
data (col. 22, lines 37-67). Imai disclose recording a plurality of sets of time (col. 6, lines 37-46). 
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Imai disclose, in fig, 1, a microphone for recording sound data, a camera for recording image 
data, and an electronic tablet for recording line drawing data (col. 4, lines 9-51) for inputting 
events. 

Regarding claim 14, Imai disclose, in fig. 1, a microphone for recording sound data, a 
camera for recording image data, and an electronic tablet for recording line drawing data (col. 4, 
lines 9-51) for inputting events. 

Regarding claim 15, Imai discloses a camera in fig. 1. Cameras inherently have lens 
systems and photoelectric conversion means. 

Regarding claim 16, Imai discloses, in fig. 26, a touch tablet (col. 20, lines 19-39). 

Regarding claim 17, Imai disclose, in fig. 8, a display section (8) for displaying input 
events (col. 8, lines 29-43) wherein a pointer indicates the relative time location within the 
continuous data (col. 10, lines 10-17). 

Regarding claim 18, Imai discloses adding new input events and updating time 
information (col. 13, lines 44-51). 

Regarding claim 19, Imai discloses that the continuous data may be digital (col. 5, lines 

5-6). 

Regarding claim 20, Imai discloses that when a new input is made, the time, an attribute 
if the type of input, and a sequence of coordinate points are written to memory (col. 6, lines 
37-67). 

Regarding claim 21, Imai, disclose an information processing apparatus in which 
continuous data which may be in the form of audio or video is stored on a disk by the continuous 
data reading section. Imai further discloses a specifying section (6) which acts as a selection 
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means for selecting data set from the data sets stored (col 5, lines 3-37). Imai further discloses, 
in fig. 35, an apparatus for playing back continuous data in which audio and video data is stored 
wherein a first audio data is represent in a first time zone (T 1) and a second audio data is 
represented in a second time (T2). The time zones are linked to overlap to produce continuous 
data (col. 22, lines 37-67). Imai further discloses a user who slides a bar on the time axis of a 
pop-up window to indicate a relative time location (col. 10, lines 10-18). 

Regarding claim 22, Imai discloses adding new input events and updating time 
information (col. 13, lines 44-51). 

Regarding claim 23, Imai discloses play back only in certain periods determined by the 
user who slides the bar on the time axis (col. 10, lines 10-14). 

Regarding claim 24, Imai discloses that when a new input is made, the time, an attribute 
if the type of input, and a sequence of coordinate points are written to memory (col. 6, lines 
37-67). 

Claims 25-36 are considered substantively equivalents to claims 1-12 respectively. 

Regarding claim 37, Imai disclose an information processing apparatus in which 
continuous data which may be in the form of audio or video is stored on a disk by the continuous 
data reading section. Imai further discloses a specifying section (6) which acts as a selection 
means for selecting data set from the data sets stored (col. 5, lines 3-37). Imai further discloses, 
in fig. 35, an apparatus for playing back continuous data in which audio and video data is stored 
wherein a first audio data is represent in a first time zone (T 1) and a second audio data is 
represented in a second time (T2). The time zones are linked to overlap to produce continuous 
data (col. 22, lines 37-67). Imai further discloses that the continuous data section may be disk 



Application/Control Number: 09/927,326 Page 7 

Art Unit: 2615 

(col. 5, lines 3-6). Imai discloses that selecting audio data may be based on a program (col. 22, 
lines 7-18). 

Regarding claim 38, Imai disclose an information processing apparatus in which 
continuous data which may be in the form of audio or video is stored on a disk by the continuous 
data reading section. Imai further discloses a specifying section (6) which acts as a selection 
means for selecting data set from the data sets stored (col. 5, lines 3 -37). Imai further discloses, 
in fig. 35, an apparatus for playing back continuous data in which audio and video data is stored 
wherein a first audio data is represent in a first time zone (Tl) and a second audio data is 
represented in a second time (T2). The time zones are linked to overlap to produce continuous 
data (col. 22, lines 37-67). Imai further discloses that the continuous data section may be disk 
(col. 5, lines 3-6). Imai discloses that selecting audio data may be based on a program (col. 22, 
lines 7-18). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co,, 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 
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2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
ornonobviousness. 

This application currently names joint inventors. In considering patentability of the 
* claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

Claims 1-3, 6-15, 18-27, and 30-38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over (USPN 6,128,037 to Anderson) in view of (USPN 5,555,098 to Parulski). 

In regards to claim 1 Anderson discloses an information processing apparatus, 
comprising: 

storage means for storing data sets having plural types of correlated data therein (e.g., 
element 354 of Fig. 3; Fig. 4; column 3, line 67 - column 4, line 6; Fig. 5; column 5, lines 14- 
32); 

selection means for selecting a data set from the data sets stored in said storage means 
(e.g., Fig. 4; column 5, lines 5-13); 

audio collecting means for collecting sound and converting said sound into audio data 
(e.g., the audio collecting means is inherent in having a sound button element 414 of Fig. 4 that 
enables a user to capture sound; column 5, lines 63-66); and 
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control means for storing audio data collected and converted by the audio collection 
means in said storage means, said control means storing said audio data correlated to the data 
selected by said selection means (e.g., column 5, line 44 - column 6, line 60). 

Anderson does not explicitly disclose nor preclude that the selected data set has first 
audio data already associated with it and that the control means stores second audio data 
correlated to the first audio data. Anderson does disclose the ability to post-annotate an image 
and discloses that the present invention allows for a user to add a recorded sound clip to any 
previously captured image (column 5, lines 38-40). 

Parulski discloses to enable any desired association between images and audio messages 
and in particular to associate multiple audio messages to a single image (column 3, lines 39-41; 
column 4, lines 21-53; column 7, lines 51-54). Therefore it would have been obvious to one of 
ordinary skill in the art at the time of the invention to have enabled Anderson's post-annotation 
with a data set that already had first audio data to further add second audio data if necessary so as 
to enable any desired association between images and audio messages as disclosed by Parulski. 

In regards to claim 2 the data set includes at least an image or audio data. 

In regards to claim 3 see Figs. 2 and 3 of Anderson and their associated description. 

In regards to claim 6 Examiner notes that Parulski discloses a controller keeps track of 
the time required to play back the entire audio message so as to enable the associated image to be 
displayed for the length of time required for completing the audio message (column 6, lines 25- 
29). Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to have added a controller to Anderson's invention to keep track of the time required to 
play back the entire audio message so as to enable the associated image to be displayed for the 
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length of time required for completing the audio message. As such, it would have further been 
obvious to one of ordinary skill in the art that when adding post-annotation sound data to a data 
set to have updated the time information indicating the time required to play back the entire 
audio message so as to enable the associated image to be displayed for the length of time 
required for completing the audio message. Further note column 6, lines 53-60 of Anderson. 

In regards to claim 7 Anderson does not explicitly disclose that the audio data is digital, 
however it is implicit within the disclosure through the disclosure that the audio data is digital. 
In particular Anderson discloses that a preferred recording medium is a flash disk which is a 
digital recording medium. As such, in order to record the audio data along with the image data . 
as disclosed by Anderson the audio data is implicitly digital. 

In regards to claim 8 Anderson discloses that after an image has been annotated it can be 
reviewed while playing back the audio clip wherein the preset time interval is the amount of time 
required to play back the audio clip. 

In regards to claim 9 see Examiners notes on the rejection of claim 1 . 

In regards to claim 10 see Examiners notes on the rejection of claim 6. 

In regards to claim 1 1 Examiner notes in Fig. 4 it is shown that audio data may be 
recorded by itself Further, Examiner notes that Anderson discloses only associating the audio 
data with a data set that is selected. As such, Anderson discloses that the second audio data is 
stored without being correlated to said first audio data when said data set is not selected. 

In regards to claim 12 see Examiners notes on the rejection of claim 6. See Fig. 4 
wherein Anderson discloses the ability to record audio clips not correlated to said first audio 
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data. Note that when it is inherent in an audio clip to have time information stored with it, 
namely the length of the audio clip is time information. 

In regards to claim 13 see Examiners notes on the rejection of claim 1 . Note that the 
microphone is implicit in Anderson's disclosure to record the sound. 

In regards to claim 14 see Examiners notes on the rejection of claim 2. 

In regards to claim 15 see Figs. 2 and 3. 

In regards to claim 18 see Examiners notes on the rejection of claim 6. 
In regards to claim 19 see Examiners notes on the rejection of claim 7. 
In regards to claim 20 see Examiners notes on the rejection of claim 8. 
In regards to claim 21 see Examiners notes on the rejection of claim 13. 
In regards to claim 22 see Examiners notes on the rejection of claim 6. 
In regards to claim 23 see Examiners notes on the rejection of claim 1 1 . 
In regards to claim 24 see Examiners notes on the rejection of claim 12. 
In regards to claims 25, 26, 27 and 30-36 see Examiners notes on the rejection of the 
claims above. 

In regards to claims 37 and 38 see Examiners notes on the rejection of the claims above. 
Note that Anderson discloses a ROM for storing program instructions (column 3, lines 64-66). 

Claims 4, 16, and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
(USPN 6,128,037 to Anderson) in view of (USPN 5,555,098 to Parulski) in view of (USPN 
5,689,742 to Chamberlain, IV). 
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In regards to claim 4 neither Anderson nor Parulski disclose means to store a line 
drawing in the data set comprising outputting means for outputting a signal, said signal 
indicating a pressed position on a pressure detection unit of predetermined surface area; and 
conversion means for converting the signal output by said outputting means into the line drawing 
data, the line drawing data being stored in said storage means. 

Chamberlain discloses annotating an image with a line drawing so that signatures and 
freehand writing can annotate and image, not just printed characters (column 2, lines 10-17). 
Chamberlain further discloses an outputting means for outputting a signal, said signal indicating 
a pressed position on a pressure detection unit of predetermined surface area in element 13 of 
Fig. 3. Chamberlain further discloses the claimed conversion means and storing the created line 
drawing in memory element 33 and further that the annotation may be digitally recorded so as to 
leave the photograph pristine (column 3, lines 17-23 and lines 61-67). Therefore it would have 
been obvious to one of ordinary skill in the art at the time of the invention to have added 
Chamberlain's annotation invention to Anderson's camera so as to enable annotating an image 
with a line drawing so that signatures and freehand writing can annotate and image, not just 
printed characters. Further, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to have recorded the annotation digitally and associated it with the image in 
as one item in a data set as suggested by Chamberlain so as to keep the photograph pristine. 
Note Andersons disclosure on column 5, lines 14-25. 

In regards to claims 16 and 28 see Examiners notes on the rejections above. 
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Claims 5, 17, and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
(USPN 6,128,037 to Anderson) in view of (USPN 5,555,098 to Parulski) in view of (JP 09- 
149307 to Kawamura etal.). ■ 

US PG-PUB 2002/0008763 to Kawamura et al. is being utilized as a translation of JP 09- 
149307 to Ejima et al. Examiner notes that the foreign application priority on the cover of the 
Kawamura reference is the application number of the above cited Japanese publication number 
to Ejima. See MPEP 901.05(111). 

In regards to claim 5 Anderson discloses display means for displaying a list of the data 
sets stored in said storage means (e.g., Fig. 4). 

Parulski discloses computing a total length of recording time of audio data belonging to 
said data sets stored in said storage means (e.g., column 6, lines 25-29). 

Neither Anderson nor Parulski disclose nor preclude displaying the total length of 
recording time of audio data on said display means. 

Kawamura discloses displaying a bar graph corresponding to the length of audio data so 
that the recording time of the recorded voice can be recognized at a glance (e.g., paragraph 0082; 
Fig. 9). Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to have displayed the total length of recording time of audio data on the display means 
in order to enable a user to recognize the recording time at a glance. 

In regards to claims 17 and 29 see Examiners notes on the rejections above. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian C. Genco who can be reached by phone at 703-305-7881 or 
by fax at 703-746-8325. The examiner can normally be reached on Monday thru Friday 8:30am 
to 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Christensen can be reached on 703-308-9644. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the customer service office whose telephone number is 703-308-4357. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Brian C Genco 
Examiner 
Art Unit 2615 



August 9, 2004 
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